Sed: Scalable & Efficient De-duplication File System for Virtual Machine Images

Virtualization is becoming widely deployed in servers to efficiently provide many logically separate execution environments by reducing the demand for physical servers, so this approach reserves physical CPU resources. Nevertheless, it still consumes large amounts of storage because each virtual machine (VM) instance, needs its own multi-gigabyte disk image. Existing systems take efforts to reduce VM image storage consumption by means of de-duplication within a storage area network (SAN) cluster. Nonetheless, a SAN cannot satisfy the increasing demand of large-scale VM hosting to cloud computing because of its cost limitation. The system proposes a SED (scalable & Efficient De-duplication) file system that has been predominantly designed for large-scale VM consumption. Its design provides hasty VM deployment with peer-to-peer (P2P) data transfer and low storage consumption by means of de-duplication on VM images. It also provides an inclusive set of storage features including on-demand fetching through a network, instant cloning for VM images, and caching with local disks by copy-on-read techniques. Experiments show that proposed system’s features perform well and introduce minor performance overhead. It shows that simply identifying zero-filled block, even in ready-to-use virtual machine disk images available internet can provide considerable savings in storage. Index Terms Cloud computing, de-duplication, file system, peer to peer data transfer, file storage, virtual machine